Non Positive SVM

Learning SVM with non positive kernels is is a problem that has been addressed in the last years but it is not really solved : indeed, either the kernel is corrected (as a pre-treatment or via a modified learning scheme), either it is used with some wellchosen parameters that lead to almost positive-definite kernels. In this work, we aim at solving the actual problem induced by non positive kernels, i.e. solving the stabilization system in the Kreı̆n space associated with the non-positive kernel. We first describe this stabilization system, then we expose a simple algorithm based on the eigen-decomposition of the kernel matrix. While providing satisfying solutions, the proposed algorithm shows limitations in terms of memory storage and computational effort. The direct resolution is still an open question. 1 Kreı̆n Space and SVM From the first stages of SVM [10] , non positive kernels are proposed and used, in particular the tanh kernel. In many application fields, some huge efforts are made to produce true Mercer kernels when the natural kernels turn out to be indefinite [4, 3]. Some author even study some kernels that are definite positive with high probablity [1]. However, until now, there is no adequate solver available. In [7, 11, 2], the authors propose to solve SVM with indefinite kernel considering that the indefinite kernel is a perturbation of a true Mercer kernel. In [5], the author states that learning with indefinite symmetric kernels is actually consisting in finding a stationary point, which is not unique but each of those performs correct separation. It has been shown [8] that learning with non positive kernel is actually solving the learning problem in a Kreı̆n space instead of a Hilbert space. It has also been shown that in this situation, the learning problem is not a minimization anymore but a stabilization problem. This means that the solution is a saddle point of the cost function. In the remaining of the section, we briefly introduce the Reproducing Kernel Kreı̆n Space (RKKS) and propose the stabilization system to be solved to train SVM in Kreı̆n space. Reproducing Kernel Kreı̆n Space Kreı̆n spaces are indefinite inner product spaces endowed with a Hilbertian topology. We recall here definitions from [8] Definition 1.1 Inner Product Let K be a vector space on the scalar field. An inner product 〈., .〉K on K is a bilinear form where for all f, g, h ∈ K, α ∈ IR : 〈f, g〉K = 〈g, f〉K 〈αf + g, h〉K = α〈f, h〉K + 〈g, h〉K 〈f, g〉K = 0, ∀g ∈ K =⇒ f = 0